Models for asynchronous message handling

To be able to benefit from the low latencies offered by recent user-level communication architectures, messageprocessing overhead must be minimized. This requirement has resulted in the development of messagehandling models that trade expressiveness for performance. We describe three models of increasing expressiveness. The least expressive model, the active messages model, disallows all blocking in message handlers. This complicates programming if messages arrive asynchronously and message handlers need to synchronize with other threads. The second model, the singlethreaded upcall model, allows blocking in some cases. It is thus more expressive than active messages, but not as expressive as the last model, popup threads, which puts no restrictions on message handlers. We have implemented all three models on the same user-level communication architecture. The implementation carefully integrates thread management and message-handling. Using this integrated system, the performance of the more expressive models is almost as good as the performance of active messages. Our work thus indicates that expressiveness need not be sacrificed for performance.